Computer simulations of language change notes
This website collects my personal notes on Computer simulations of language change. These notes are provided to bring full transparency to my research process. Of course, since they are only notes, they do not reflect my final thoughts on a topic, and should not be interpreted as such. To read finished papers, please consult my website. Do not use these notes as a basis for your own scientific research. Start from high-quality, peer-reviewed scientific literature instead.
Lorenz middle run
↓
↳ sensitive dependence on initial conditions
↕
Western science
From nearly the same starting point, Edward Lorenz saw his computer weather produce patterns that grew farther and farther apart until all resemblance disappeared.
(From Lorenz’s 1961 printouts)
↓
Butterfly effect
↓ ≈
aperiodic system
Examples of aperiodic systems
If the weather ever did reach a state exactly like one it had reached before, every gust and cloud the same, then presumably it would repeat itself forever after and the problem of forecasting would become trivial.
chaos from a deterministic system
graphic images are key
differential equations
↓ allow for
parameters at the start
↓ therefore …
topology and dynamical systems
these are loss surfaces, but the idea is the same
| Stale’s stability | any chaotic system |
|---|---|
| a system can behave erratically, but the erratic behaviour cannot be stable | a system which can have chaos and stability at once |
| stable behaviour that does not disappear just because some number was changed a tiny bit ↓ strangeness does not go away when perturbed by noise |
|
The difference is that Smale assumed that behaviour we see in the real world is actually stable, since real life has many disturbances and noise. In reality, systems are chaotic by definition, since stable systems would be those that are disturbed by tiny differences. It is a philosophical discussion, in that sense.
ecology
models as ‘caricatures’
discrete time intervals
feedback loop function
Feedback can get out of hand, as it does when sound from a loudspeaker feeds back through a microphone and is rapidly amplified to an unbearable shriek. Or feedback can produce stability, as a thermostat does in regulating the temperature of a house: any temperature above a fixed point leads to cooling, and any temperature below it leads to heating.
adding realism
logistic equation
a population reaches equilibrium after rising, overshooting, and falling back
erratic behaviour
supremacy of chaos
bifurcation diagram
chaos region
↓ thus…
chaos is also regular
↓
regularity - chaos - regularity - chaos …
In the real world, a researcher sees only a single point, with no knowledge of the dynamics of the system. He would only see one kind of behaviour – possibly a steady state, possibly a seven-year cycle, possibly apparent randomness. He would have no way of knowing that the same system, with some slight change in some parameter, could display patterns of a completely different kind.
In any one-dimensional system, if a regular cycle of period three ever appears, the same system will also display regular cycles of every other length, as well as completely chaotic cycles.
- James Yorke
The outline of the bifurcation diagram as May first saw it, before more powerful computation revealed its rich structure.
cantor set
Cantor dust
Begin with a line; remove the middle third; then remove the middle third of the remaining segments; and so on. The Cantor set is the dust of points that remains. They are infinitely many, but their total length is 0.
Mandelbrot saw the Cantor set as a model for the occurrence of errors in an electronic transmission line. Engineers saw periods of error-free transmission, mixed with periods when errors would come in bursts. Looked at more closely, the bursts, too, contained error-free periods within them. And so on—it was an example of fractal time. At every time scale, from hours to seconds, Mandelbrot discovered that the relationship of errors to clean transmission remained constant. Such dusts, he contended, are indispensable in modeling intermittency.
fractal
A fractal shore
A computer-generated coastline: the details are random, but the fractal dimension is constant, so the degree of roughness or irregularity looks the same no matter how much the image is magnified.
How big is it? How long does it last? These are the most basic questions a scientist can ask about a thing. They are so basic to the way people conceptualize the world that it is not easy to see that they imply a certain bias. They suggest that size and duration, qualities that depend on scale, are qualities with meaning, qualities that can help describe an object or classify it. When a biologist describes a human being, or a physicist describes a quark, how big and how long are indeed appropriate questions. In their gross physical structure, animals are very much tied to a particular scale. Imagine a human being scaled up to twice its size, keeping all proportions the same, and you imagine a structure whose bones will collapse under its weight. Scale is important.
The physics of earthquake behavior is mostly independent of scale. A large earthquake is just a scaled-up version of a small earthquake. That distinguishes earthquakes from animals, for example—a ten-inch animal must be structured quite differently from a one-inch animal, and a hundred-inch animal needs a different architecture still, if its bones are not to snap under the increased mass. Clouds, on the other hand, are scaling phenomena like earthquakes. Their characteristic irregularity—describable in terms of fractal dimension—changes not at all as they are observed on different scales. That is why air travelers lose all perspective on how far away a cloud is. Without help from cues such as haziness, a cloud twenty feet away can be indistinguishable from two thousand feet away. Indeed, analysis of satellite pictures has shown an invariant fractal dimension in clouds observed from hundreds of miles away.
scaling phenomena
↓
claim of fractal geometry
turbulence
(picture from The Constructor)
All the rules seem to break down. When flow is smooth, or laminar, small disturbances die out. But past the onset of turbulence, disturbances grow catastrophically.
Think for example of when you pour water in a glass. If you do it slowly, the glass is filled in an ‘undisturbed’ way. However, when you really pour in the water, you get a turbulent flow of water where water molecules are scattered around violently.
Flow between rotating cylinders
The patterned flow of water between two cylinders gave Harry Swinney and Jerry Gollub a way to look at the onset of turbulence. As the rate of spin is increased, the structure grows more complex. First the water forms a characteristic pattern of flow resembling stacked doughnuts. Then the doughnuts begin to ripple. The physicists used a laser to measure the water’s changing velocity as each new instability appeared.
phase space
Pendulum
In this video, pendulum movement is shown with 𝑣 = velocity and θ = angle. The ground plane shows the phase space.
However, if we add realism (friction), both the angle and velocity will converge towards zero, which means the centre point of the ground plane is an ‘attractor’.
utility of phase space
multi-dimensional phase spaces
periodic attractor / limit cycle
non-periodic phase space
Poincaré map / return map
Exposing an attractor’s structure
The strange attractor above—first one orbit, then ten, then one hundred—depicts the chaotic behavior of a rotor, a pendulum swinging through a full circle, driven by an energetic kick at regular intervals. By the time 1,000 orbits have been drawn (below), the attractor has become an impenetrably tangled skein. To see the structure within, a computer can take a slice through an attractor, a so-called Poincaré section. The technique reduces a three-dimensional picture to two dimensions. Each time the trajectory passes through a plane, it marks a point, and gradually a minutely detailed pattern emerges. This example has more than 8,000 points, each standing for a full orbit around the attractor. In effect, the system is “sampled” at regular intervals. One kind of information is lost; another is brought out in high relief.
As far as I can tell, the strange attractor is not a single point as with the pendulum example. Rather, it is a multidimensional phenomenon.
Hénon found that the oversimplification paid off. By abstracting only the essence of his system, he made discoveries that applied to other systems as well, and more important systems.
Hénon equation
attractor
This is why the starting conditions do not matter: as long as the starting point lies somewhere near the attractor, the next few points will converge to the attractor with great rapidity.
The attractor of Hénon
A simple combination of folding and stretching produced an attractor that easy to compute yet still poorly understood by mathematicians. As thousands, the millions of points appear, more and more detail emerges. What appear to be single lines prove, on magnification, to be pairs, then pairs of pairs. Yet whether any two successive points appear nearby or far apart is unpredictable.
universality
intransitive system
An observer might see one kind of behavior over a very long time, yet a completely different kind of behavior could be just as natural for the system.
Clock example
In a trivial way, a standard pendulum clock is an intransitive system. A steady flow of energy comes in from a wind-up spring or a battery through an escapement mechanism. A steady flow of energy is drained out by friction. The obvious equilibrium state is a regular swinging motion. If a passerby bumps the clock, the pendulum might speed up or slow down from the momentary jolt but will quickly return to its equilibrium. But the clock has a second equilibrium as well—a second valid solution to its equations of motion—and that is the state in which the pendulum is hanging straight down and not moving.
almost-intransistive system
| intransitive system | almost-intransitive system |
|---|---|
| multiple equilibria (attractors) are possible, but change lies in parameters | multiple equilibria are possible within the same parameter settings |
An intransitive system is characterized by two or more mutually coexisting attractors. Which attractor will be traversed is determined by the initial state of the system. In contrast, the long-term dynamical characteristics of transitive systems are not affected by the initial state as there is only one attractor.
Universality theory
Universality offered the hope that by solving an easy problem physicists could solve much harder problems.
It’s not an academic question any more to ask what’s going to happen to a cloud. People very much want to know—and that means there’s money available for it. That problem is very much within the realm of physics and it’s a problem very much of the same caliber. You’re looking at something complicated, and the present way of solving it is to try to look at as many points as you can, enough stuff to say where the cloud is, where the warm air is, what its velocity is, and so forth. Then you stick it into the biggest machine you can afford and you try to get an estimate of what it’s going to do next. But this is not very realistic.
- Feigenbaum
This also echoes the FCG arguments by Katrien Beuls and Paul Van Eecke on language acquisition and language models. Fair enough.
I truly do want to know how to describe clouds. But to say there’s a piece over here with that much density, and next to it a piece with this much density—to accumulate that much detailed information, I think is wrong. It’s certainly not how a human being perceives those things, and it’s not how an artist perceives them. Somewhere the business of writing down partial differential equations is not to have done the work on the problem.
- Feigenbaum
non-linearity
turbulence onset
↓
very small change in temperature
Another view into bifurcation
When an experiment like Libchaber’s convection cell produces a steady oscillation, its phase-space portrait is a loop, repeating itself at regular intervals (top left). An experimenter measuring the frequencies in the data will see a spectrum diagram with a strong spike for this single rhythm. After a period-doubling bifurcation, the system loops twice before repeating itself exactly (center), and now the experimenter sees a new rhythm at half the frequency—twice the period—of the original. New period-doublings fill in the spectrum diagram with more spikes.
bifurcation and looping systems
‘era of computer experimentation’
The modifications, the compromises, the approximations needed to digitize systems of nonlinear differential equations were too suspect. Simulations break reality into chunks, as many as possible but always too few. A computer model is just a set of arbitrary rules, chosen by programmers.
Whenever a good physicist examines a simulation, he must wonder what bit of reality was left out, what potential surprise was sidestepped. Libchaber liked to say that he would not want to fly in a simulated airplane—he would wonder what had been missed. Furthermore, he would say that computer simulations help to build intuition or to refine calculations, but they do not give birth to genuine discovery.
An assortment of Julia sets
Julia set
Mandelbrot set calculation
Pinball analogy
Like most pinball machines it has a plunger with a spring. You pull back the plunger and release it to send the ball up into the playing area. The machine has the customary tilted landscape of rubber edges and electric bouncers that give the ball a kick of extra energy. The kick is important: it means that energy does not just decay smoothly. For simplicity’s sake this machine has no flippers at the bottom, just two exit ramps. The ball must leave by one ramp or the other.
This is deterministic pinball—no shaking the machine. Only one parameter controls the ball’s destination, and that is the initial position of the plunger. Imagine that the machine is laid out so that a short pull of the plunger always means that the ball will end up rolling out the right-hand ramp, while a long pull always means that the ball will finish in the left-hand ramp. In between, the behavior gets complex, with the ball bouncing from bumper to bumper in the usual energetic, noisy, and variably long-lived manner before finally choosing one exit or the other.
Now imagine making a graph of the result of each possible starting position of the plunger. The graph is just a line. If a position leads to a righthand departure, plot a red point, and plot a green point for left. What can we expect to find about these attractors as a function of the initial position? The boundary proves to be a fractal set, not necessarily self-similar, but infinitely detailed. Some regions of the line will be pure red or green, while others, when magnified, will show new regions of red within the green, or green within the red. For some plunger positions, that is, a tiny change makes no difference. But for others, even an arbitrarily small change will make the difference between red and green.
To add a second dimension meant adding a second parameter, a second degree of freedom. With a pinball machine, for example, one might consider the effect of changing the tilt of the playing slope. One would discover a kind of in-and–out complexity that would give nightmares to engineers responsible for controlling the stability of sensitive, energetic real systems with more than one parameter—electrical power grids, for example, and nuclear generating plants, both of which became targets of chaos-inspired research in the 1980s. For one value of parameter A, parameter B might produce a reassuring, orderly kind of behavior, with coherent regions of stability. Engineers could make studies and graphs of exactly the kind their linear-oriented training suggested. Yet lurking nearby might be another value of parameter A that transforms the importance of parameter B.
A warning about complex behaviour
To researchers and engineers, there was a lesson in these pictures—a lesson and a warning. Too often, the potential range of behavior of complex systems had to be guessed from a small set of data. When a system worked normally, staying within a narrow range of parameters, engineers made their observations and hoped that they could extrapolate more or less linearly to less usual behavior. But scientists studying fractal basin boundaries showed that the border between calm and catastrophe could be far more complex than anyone had dreamed. “The whole electrical power grid of the East Coast is an oscillatory system, most of the time stable, and you’d like to know what happens when you perturb it,” Yorke said. “You need to know what the boundary is. The fact is, they have no idea what the boundary looks like.”
Barnsley chaos game
limited set of rules could create an infinitely complex shape
strange attractors
“orderly disorder created by simple processes”
What’s actually the case is that, as physicians or scientists learning all 50,000 parts of everything, we resent the possibility that there are in fact universal elements of motion. And Bernardo comes up with one and look what happens.
The choice is always the same. You can make your model more complex and more faithful to reality, or you can make it simpler and easier to handle. Only the most naïve scientist believes that the perfect model is the one that perfectly represents reality. Such a model would have the same drawbacks as a map as large and detailed as the city it represents, a map depicting every park, every street, every building, every tree, every pothole, every inhabitant, and every map. Were such a map possible, its specificity would defeat its purpose: to generalize and abstract. Mapmakers highlight such features as their clients choose. Whatever their purpose, maps and models must simplify as much as they mimic the world.
“universal laws guding the behaviour of feedback functions”
📖 Dynamics Of Complex Systems
Computer simulations of language change notes
This website collects my personal notes on Computer simulations of language change. These notes are provided to bring full transparency to my research process. Of course, since they are only notes, they do not reflect my final thoughts on a topic, and should not be interpreted as such. To read finished papers, please consult my website. Do not use these notes as a basis for your own scientific research. Start from high-quality, peer-reviewed scientific literature instead.